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Introduction 

Every three years the focus of the international education community shifts to the release of the 
Programme for International Student Assessment (PISA) study conducted by the Organisation for 
Economic Co-operation and Development (OECD). No other international study of education 
commands as much attention as PISA. Numerous media events sponsored by the OECD as well 
as national education ministries take place around the world for the release of the latest results. 

In an age of intense global competition among information-based economies where education is 
increasingly viewed as an important (if not the most important) national “competitive advantage”, 
in less than two decades, PISA has become the widely used if inadequate proxy for the 
performance of education systems around the world. The OECD’s interest in education in the 
service of human capital development and economic growth is of course not surprising given the 
organization’s focus. The expansion of PISA to include the assessment of financial literacy 
demonstrates the narrow goals driving the program. 

The number of participating countries/economies grows with each testing cycle - countries 
increasingly feel they cannot be left out of this influential test. Andreas Schleicher who oversees 
PISA remarks that, “in a global economy, the benchmark for educational success is no longer 
improvement by national standards alone, but by the best performing school systems 
internationally.” 

PISA 2012 tested over half a million 15-year-old students in 65 participating countries and 
economies (between 5,000 and 10,000 students from at least 150 schools were tested in each 
country). The major focus of the survey was mathematics, with a secondary focus on reading, 
science and problem-solving. As noted, financial literacy was included for the first time in 2012. 
The assessment is a paper-based test lasting 2 hours with an additional 40 minutes added in 
some countries for a computer-based assessment of math, reading and problem-solving. PISA 
2015 is expected to transition to a fully computer-based assessment. 


Highlights of Canadian PISA results 

Canada’s participation in PISA is coordinated through a partnership between the Council of 
Ministers of Education Canada (CMEC), Employment and Social Development Canada (ESDC), 
and Statistics Canada. Approximately 21,000 Canadian students from about 900 schools in all 
provinces were tested in the spring of 2012. A companion report on the Canadian PISA results, 
Measuring Up: Canadian Results of the OECD PISA Study, was released on the same day as the 
OECD report. 
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According to the report these are some key findings for Canada from PISA 201 2 (major domain of 
mathematics; minor domains of reading and science): 

• As in past PISA cycles, Canadian 15-year-olds placed well above the OECD average and 
remain among the top performers in mathematics. Only nine participating 
countries/economies performed better than Canada: Shanghai-China, Singapore, Hong 
Kong-China, Chinese Taipei, Korea, Macao-China, Japan, Liechtenstein, and Switzerland. 

• Reading and science results also put Canada in the top tier of participating countries and 
economies, well above the OECD average. 

• Canada continues to stand out as one of the high-performing countries with relatively high 
equity in student performance (measured by the gap between the highest- and lowest- 
performing students). In all three areas assessed by PISA, the gap between high- and 
low-achievers is smaller in Canada than in OECD countries on average. 

• Although overall Canadian scores in mathematics remain high, they have drifted 
downward over time. There is a clear trend showing a decrease in average score in most 
provinces, as well as an increase in the number of countries outperforming Canada. 
Science scores also showed a decline over the years while reading scores remained 
generally at the same level. 

• On average across Canada, there was significant variation in mathematics performance 
according to gender, with boys outperforming girls. This pattern was similar in most other 
participating countries. In science, the performance of boys and girls was similar. 

• In reading, girls were still well ahead of boys in Canada and internationally. The gap 
between boys and girls was smaller for students who did the assessment on computer 
(reading of digital texts). 

• In mathematics, Canadian results showed some differences by language of the school 
system. In most cases, students attending majority-language school systems 
outperformed students in minority-language school systems. 


Obsessive focus on rankings 

As an article about PISA in The Economist e arlier this year aptly put it, “international league 
tables have acquired a central role in debates about education policy.” 

In her commentary on the PISA 2009 results, Jodene Dunleavy, Senior Policy Analyst for the 
Nova Scotia Department of Education, questions the intense focus on the international rankings, 
likening them to the “Education Olympics”: 

I’d like to put some of the blame for public reaction to PISA scores on the OECD, itself. 

It’s easy to feel intimidated by the volume of figures and explanations that flow from each 
assessment. But this alone cannot explain the overwhelming amount of attention paid to a 
single, league-style table ranking the 65 participating countries on combined reading, 
mathematics, and scientific literacy scores. Witnessing how results get taken up in the 
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public domain, it is hard not to feel that the PISA country rankings have become the 
Olympics of the education world. 

The continued focus on PISA rankings by the media and politicians has resulted in the quality of 
education systems being reduced to a simplistic matter of performing better or worse on the test. 
Educators are well aware of the adverse consequences of ranking and publishing test results as 
this recent report by People for Education emphasizes: 

The most controversial use of test score data has been to create rankings or league 
tables, either by governments, in some cases, or by advocacy groups. Some believe that 
school rankings can be used to improve performance, or as useful information to promote 
choice for parents as consumers making decisions about their children’s schools. Others 
have argued that rankings are a simplistic and often inaccurate way of judging quality in 
schools. They say rankings can have a corrosive effect on student and teacher morale, 
and that the public nature of the information sharing can redouble the narrowing 
tendencies of the measures. Critics have also argued these rankings reinforce negative 
perceptions about schools in socio-economically challenged areas, and that this type of 
high-stakes or high-visibility assessment can lead, paradoxically, to unintended 
consequences, such as “gaming the system” by, for example, focusing efforts on ‘bubble 
children’, those whose performance is near the cut-score for proficiency as opposed to 
lower- and higher-performing children where changes in performance will not affect the 
school’s standing. 

Educators have consistently resisted the idea that the results of large-scale assessments, 
particularly in a narrow band of subjects, reflect the range of their responsibilities and 
goals, and they have strenuously objected to the use of numbers to judge performance 
when they have little control over the conditions under which they work. (pp. 17-18) 

In an attempt to broaden the Canadian definition of school success beyond the results of external 
large-scale assessments such as PISA, Ontario-based People for Education has started an 
ambitious five-year project to engage parents, teachers, policy-makers and community members 
in identifying and expanding the indicators used to evaluate school success - in essence 
examining what really matters in public education and then looking for ways to measure it. 


Tackling inequity within and outside the education system 

Dunleavy says that a number of issues are often left out of discussions about the PISA results 
such as the extent to which countries vary in terms of matching high performance with equity of 
educational outcomes, something Canada consistently excels at, even with its highly diverse 
population. Canada has smaller equity gaps in education than many countries, in part due to 
Canadian values around justice, equity and inclusion, and our social safety net, albeit eroded. 
Our system of social programs attempts to mitigate the impacts of inequities on children, families 
and schools. 


3 


In what they describe as the “Canadian Way”, Hargreaves and Shirley (2012) believe that our 
educational success is considerably more complex than can be adequately explained by any 
individual jurisdiction’s particular short-term education policies. They conclude that it has more 
to do with 

constellations of policies that run across provinces and systems, accumulate over time, 
and are consistent with a longstanding culture of high regard for public education, strong 
support for the teaching profession, and broadly collaborative and inclusive processes of 
educational change management, inspired by sets of commonly shared beliefs. This 
embedded and inclusive Canadian Way - that is being threatened by the global trend to 
weaken district involvement and control in favour of more and more centralized direction - 
says more about Canada as a society than it does about the relative value of any specific 
provincial policy. 

(p. 13) 

While there is much we can and must do to further improve the education system in Canada in 
terms of quality and equity, schools cannot do it alone. In addition to improving Aboriginal 
education, we need to address the many issues, both within and outside schools, that hinder 
student learning and prevent all students from doing their best including child poverty and income 
inequality, school climate issues such as bullying/cyberbullying, mental health problems, class 
size and composition issues, and lack of resources in the schools to name a few. 


Educational success, the teaching profession and teacher unions 

If we’re looking for direction on education policy from outside our borders, many educators 
believe it makes sense to look towards countries like Finland. The Finnish approach to education 
reform is the anti-thesis of the global education reform movement (known as GERM) evident in 
the U.S. and elsewhere. GERM is characterized by competition, standardization, high-stakes 
testing, privatization and other market-based education reforms. 

In her review of Finnish Lessons: What Can the World Learn from Educational Change in 
Finland? (2011) by education expert Pasi Sahlberg, Vivian McCaffrey notes that, “what is most 
perplexing for international experts is that Finland has produced top-performing students while 
eschewing market-based education reforms premised on competition and standardized tests.” 
She quotes Sahlberg: “Finland is an example of a nation that lacks school inspection, 
standardized curriculum, high-stakes student assessments, test-based accountability, and a race- 
to-the-top mentality with regard to educational change”. 

The important link between educational success and the quality of the teaching profession is 
something Finns are well aware of. According to McCaffrey (2011), the “centrepiece of Finnish 
education is the nation’s teaching force”, particularly the high societal status of teachers and 
teaching (teachers are highly respected in their communities), strong public trust in the 
profession, rigorous selection and training, and a focus on teacher professional development and 
autonomy. 

While there is much to learn from the Finnish education model, Sahlberg cautions against 
transplanting this model in other countries, noting that “Finland's success is a result of finding its 
own way of change rather than doing more of the same.” 
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Similar to Finland, Canada’s strong public education system is accompanied by a high quality 
teaching profession and strong teacher unions. We attract strong teacher candidates, support 
and develop teachers over the course of their careers, and like other high performing countries 
we have strong teacher unions whose protection of salaries and working conditions, emphasis on 
effective ongoing professional development and learning, and focus on improving the conditions 
for teaching and learning are essential to recruiting and retaining strong teachers. 


PISA’s impact on education policy 

Through its triennial PISA reports and other mechanisms such as the annual Education at a 
Glance reports, the OECD exerts a powerful influence on the direction of global education policy. 
The OECD’s stated intent through PISA is to improve education systems, through what can be 
described as “education governance by comparison”, for the purpose of enabling countries to 
better compete in the global economy. It is a narrow perspective on the purpose of education. 

Research shows that PISA results are having a growing influence on education policy reform in 
the majority of participating countries. In his analysis of the policy impact of PISA, Breakspear 
states that “PISA is becoming an influential element of education policy-making processes at the 
national level. Furthermore, the findings provide preliminary evidence that PISA is being used and 
integrated within national/federal policies and practices of assessment and evaluation, curriculum 
standards and performance targets.” 

There is evidence that PISA is being used as a policy lever in Canada. Breakspear notes that 
“some Canadian provinces have expressed PISA-related performance targets in general terms, 
such as ‘improving PISA score’ or ‘improving PISA ranking’.” (p. 26) In addition, provinces 
monitor PISA results to validate their own provincial assessment results, and Canada’s national 
assessment program was redesigned several years ago to harmonize with PISA in terms of 
target population and subject areas covered. 

We need to do a better job of articulating what we can, and can’t, learn from the PISA study as 
well as continue to raise concerns over the use and misuse of the results (Dunleavy, 2011). As 
we’ve noted for previous rounds of PISA testing, PISA’s snapshot of how education systems 
perform in the areas of numeracy, reading and science conveys neither the complexity nor the 
breadth of these education systems. 
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